A Study about Heterogeneous Network Issues Management based on Enhanced Inter-cell Interference Coordination and Machine Learning Algorithms

Under the circumstance of fast growing demands for mobile data, Heterogeneous Networks (HetNets) has been considered as one of the key technologies to solve 1000 times mobile data challenge in the coming decade. Although the unique multi-tier topology of HetNets has achieved high spectrum efficiency and enhanced Quality of Service (QoS), it also brings a series of critical issues. In this thesis, we present an investigation on understanding the cause of HetNets challenges and provide a research on state of arts techniques to solve three major issues: interference, offloading and handover. The first issue addressed in the thesis is the cross-tier interference of HetNets. We introduce Almost Blank Subframes (ABS) to free small cell UEs from cross-tier interference, which is the key technique of enhanced Inter-Cell Interference Coordination (eICIC). Nash Bargain Solution (NBS) is applied to optimize ABS ratio and UE partition. Furthermore, we propose a power based multi-layer NBS Algorithm to obtain optimal parameters of Further enhanced Inter-cell Interference Coordination (FeICIC), which significantly improve macrocell efficiency compared to eICIC. This algorithm not only introduces dynamic power ratio but also defined opportunity cost for each layer instead of conventional zero-cost partial fairness. Simulation results show the performance of proposed algorithm may achieve up to 31.4% user throughput gain compared to eICIC and fixed power ratio FeICIC. This thesis’ second focusing issue is offloading problem of HetNets. This includes (1) UE offloading from macro cell and (2) small cell backhaul offloading. For first aspect, we have discussed the capability of machine learning algorithms tackling this challenge and propose the User-Based K-means Algorithm (UBKCA). The proposed algorithm establishes a closed loop Self-Organization system on our HetNets scenario to maintain desired offloading factor of 50%, with cell edge user factor 17.5% and CRE bias of 8dB. For second part, we further apply machine learning clustering method to establish cache system, which may achieve up to 70.27% hit-ratio and reduce request latency by 60.21% for Youtube scenario. K-Nearest Neighbouring (KNN) is then applied to predict new users’ content preference and prove our cache system’s suitability. Besides that, we have also proposed a system to predict users’ content preference even if the collected data is not complete. The third part focuses on offloading phase within HetNets. This part detailed discusses CRE’s positive effect on mitigating ping-pong handover during UE offloading, and CRE’s negative effect on increasing cross-tier interference. And then a modified Markov Chain Process is established to map the handover phases for UE to offload from macro cell to small cell and vice versa. The transition probability of MCP has considered both effects of CRE so that the optimal CRE value for HetNets can be achieved, and result for our scenario is 7dB. The combination of CRE and Handover Margin is also discussed

On the Convergence of Deep Learning with Differential Privacy

In deep learning with differential privacy (DP), the neural network achieves the privacy usually at the cost of slower convergence (and thus lower performance) than its non-private counterpart. This work gives the first convergence analysis of the DP deep learning, through the lens of training dynamics and the neural tangent kernel (NTK). Our convergence theory successfully characterizes the effects of two key components in the DP training: the per-sample clipping (flat or layerwise) and the noise addition. Our analysis not only initiates a general principled framework to understand the DP deep learning with any network architecture and loss function, but also motivates a new clipping method -- the global clipping, that significantly improves the convergence while preserving the same privacy guarantee as the existing local clipping. In terms of theoretical results, we establish the precise connection between the per-sample clipping and NTK matrix. We show that in the gradient flow, i.e., with infinitesimal learning rate, the noise level of DP optimizers does not affect the convergence. We prove that DP gradient descent (GD) with global clipping guarantees the monotone convergence to zero loss, which can be violated by the existing DP-GD with local clipping. Notably, our analysis framework easily extends to other optimizers, e.g., DP-Adam. Empirically speaking, DP optimizers equipped with global clipping perform strongly on a wide range of classification and regression tasks. In particular, our global clipping is surprisingly effective at learning calibrated classifiers, in contrast to the existing DP classifiers which are oftentimes over-confident and unreliable. Implementation-wise, the new clipping can be realized by adding one line of code into the Opacus library

Practice with Graph-based ANN Algorithms on Sparse Data: Chi-square Two-tower model, HNSW, Sign Cauchy Projections

Sparse data are common. The traditional ``handcrafted'' features are often sparse. Embedding vectors from trained models can also be very sparse, for example, embeddings trained via the ``ReLu'' activation function. In this paper, we report our exploration of efficient search in sparse data with graph-based ANN algorithms (e.g., HNSW, or SONG which is the GPU version of HNSW), which are popular in industrial practice, e.g., search and ads (advertising). We experiment with the proprietary ads targeting application, as well as benchmark public datasets. For ads targeting, we train embeddings with the standard ``cosine two-tower'' model and we also develop the ``chi-square two-tower'' model. Both models produce (highly) sparse embeddings when they are integrated with the ``ReLu'' activation function. In EBR (embedding-based retrieval) applications, after we the embeddings are trained, the next crucial task is the approximate near neighbor (ANN) search for serving. While there are many ANN algorithms we can choose from, in this study, we focus on the graph-based ANN algorithm (e.g., HNSW-type). Sparse embeddings should help improve the efficiency of EBR. One benefit is the reduced memory cost for the embeddings. The other obvious benefit is the reduced computational time for evaluating similarities, because, for graph-based ANN algorithms such as HNSW, computing similarities is often the dominating cost. In addition to the effort on leveraging data sparsity for storage and computation, we also integrate ``sign cauchy random projections'' (SignCRP) to hash vectors to bits, to further reduce the memory cost and speed up the ANN search. In NIPS'13, SignCRP was proposed to hash the chi-square similarity, which is a well-adopted nonlinear kernel in NLP and computer vision. Therefore, the chi-square two-tower model, SignCRP, and HNSW are now tightly integrated

Spatial Scattering Modulation with Multipath Component Aggregation Based on Antenna Arrays

In this paper, a multipath component aggregation (MCA) mechanism is introduced for spatial scattering modulation (SSM) to overcome the limitation in conventional SSM that the transmit antenna array steers the beam to a single multipath (MP) component at each instance. In the proposed MCA-SSM system, information bits are divided into two streams. One is mapped to an amplitude-phase-modulation (APM) constellation symbol, and the other is mapped to a beam vector symbol which steers multiple beams to selected strongest MP components via an MCA matrix. In comparison with the conventional SSM system, the proposed MCA-SSM enhances the bit error performance by avoiding both low receiving power due to steering the beam to a single weak MP component and inter-MP interference due to MP components with close values of angle of arrival (AoA) or angle of departure (AoD). For the proposed MCA-SSM, a union upper bound (UUB) on the average bit error probability (ABEP) with any MCA matrix is analytically derived and validated via Monte Carlo simulations. Based on the UUB, the MCA matrix is analytically optimized to minimize the ABEP of the MCA-SSM. Finally, numerical experiments are carried out, which show that the proposed MCA-SSM system remarkably outperforms the state-of-the-art SSM system in terms of ABEP under a typical indoor environment

K-BERT: Enabling Language Representation with Knowledge Graph

Pre-trained language representation models, such as BERT, capture a general language representation from large-scale corpora, but lack domain-specific knowledge. When reading a domain text, experts make inferences with relevant knowledge. For machines to achieve this capability, we propose a knowledge-enabled language representation model (K-BERT) with knowledge graphs (KGs), in which triples are injected into the sentences as domain knowledge. However, too much knowledge incorporation may divert the sentence from its correct meaning, which is called knowledge noise (KN) issue. To overcome KN, K-BERT introduces soft-position and visible matrix to limit the impact of knowledge. K-BERT can easily inject domain knowledge into the models by equipped with a KG without pre-training by-self because it is capable of loading model parameters from the pre-trained BERT. Our investigation reveals promising results in twelve NLP tasks. Especially in domain-specific tasks (including finance, law, and medicine), K-BERT significantly outperforms BERT, which demonstrates that K-BERT is an excellent choice for solving the knowledge-driven problems that require experts.Comment: 8 pages, 2019091

Missile-Borne SAR Raw Signal Simulation for Maneuvering Target

SAR raw signal simulation under the case of maneuver and high-speed has been a challenging and urgent work recently. In this paper, a new method based on one-dimensional fast Fourier transform (1DFFT) algorithm is presented for raw signal simulation of maneuvering target for missile-borne SAR. Firstly, SAR time-domain raw signal model is given and an effective Range Frequency Azimuth Time (RFAT) algorithm based on 1DFFT is derived. In this algorithm, the “Stop and Go” (SaG) model is adopted and the wide radar scattering characteristic of target is taken into account. Furthermore, the “Inner Pulse Motion” (IPM) model is employed to deal with high-speed case. This new RFAT method can handle the maneuvering cases, high-speed cases, and bistatic radar cases, which are all possible in the missile-borne SAR. Besides, this raw signal simulation adopts the electromagnetic scattering calculation so that we do not need a scattering rate distribution map as the simulation input. Thus, the multiple electromagnetic reflections can be considered. Simulation examples prove the effectiveness of our method